Interactive artificial intelligence

ABSTRACT

Behavior of a device is modified based on the device&#39;s experience. The device includes: (i) a sensing unit for sensing signals; (ii) a concern-generating unit programmed to generate concern-parameters; (iii) an emotion-generating unit programmed to generate emotion-parameters; and (iv) an actuating unit for actuating the device. When the device is in a situation, the device extracts memory relevant to the situation to obtain concern-parameters previously generated in the situation. The behavior of the device is regulated by concern-parameters in the memory and emotion-parameters generated based on the concern-parameters, and accordingly, the device can modify or improve its behavior.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a method for adjusting behaviorof a device, and particularly to a method for adjusting the behaviorbased on the device's experience.

[0003] 2. Description of the Related Art

[0004] Conventionally, various controlling methods have been availablefor controlling a device in accordance with a user's demand.

[0005] In such controlling methods, normally, the user sets a targetvalue at output which the user wants, and the device is controlled insuch a way that the output matches the target value, while feeding theoutput back to a control system which compares the feedback and thetarget value to adjust the output. In the above, by feeding the outputback to the system to adjust the output, the output of the device to becontrolled can approach the target value, thereby achieving controlsatisfying the user's preference.

[0006] However, the above control system may not be appropriate in anarea where the user can develop emotions such as companionship toward adevice. Such a device includes toys, games, communication tools, searchtools, and other tools and devices subjected to personal use. Further,if a target value is not determinable due to the system's complexity orlack of information, the conventional control system does not work.

[0007] Furthermore, if a device senses numerous pieces of information,it is difficult to sort them out and use them as useful information toact thereon. If the device is operated solely in accordance with theuser's command, the above may not be a problem. However, for anautonomic device which acts without the user's command, it is importantto obtain meaningful information.

SUMMARY OF THE INVENTION

[0008] A general objective of the present invention is to provide acontrol system which enables a device to autonomously modify itsbehavior or performance through interaction with its externalenvironment, i.e., based on its experience. That is a self-developmentsystem. The control system can generate pseudo-emotions which are usedas a parameter for controlling behavior. The control system can collectinformation on an object of its own concern, and can store theinformation and update it by itself. This autonomous behavior system isadvantageous, especially when applied to robots, toys, or games. Thepresent invention has exploited a real-time basis behavior-adjustingsystem.

[0009] An embodiment of the present invention is a method for adjustingbehavior of a device based on the device's experience. The devicecomprises: (i) a sensing unit for sensing signals; (ii) aconcern-generating unit programmed to generate concern-parameters; (iii)an emotion-generating unit programmed to generate emotion-parameters;and (iv) an actuating unit for actuating the device. The methodcomprises the steps of: (a) recognizing an object based on signalsreceivable by the device, said device having initial concern parameters;(b) extracting information, if any, on concern-parameters from a memoryunder at least the index of the object, said memory storing under theindex of the object, information on concern-parameters previouslygenerated by the device through past interaction with the object; (c)modifying the initial concern-parameters by the extracted information onconcern-parameters; (d) actuating the device based on the modifiedconcern- parameters and emotion-parameters generated by the device basedat least on the modified concern-parameters; and (e) inputting in thememory, under the index of the object, concern-parameters generated bythe device upon interaction with the object, thereby updating thememory. Accordingly, when the device is in a situation, the devicerecalls (extracts) memory relevant to the situation to obtainconcern-parameters previously generated in the situation. The behaviorof the device is regulated by concern-parameters in the memory andemotion-parameters generated based on the concern-parameters, andaccordingly, the device can modify or improve its behavior.

[0010] In the above, in an embodiment, the emotion-parameters aregenerated based on a discrepancy between the current concern-parametersand the modified concern-parameters under predetermined rules. By usinga hierarchical structure composed of a concern layer and a emotionlayer, wherein the former is higher than the latter, the device canbehave autonomously. Further, in an embodiment, the behavior is selectedbased on the modified concern-parameters, and is then modified based onthe emotion-parameters under predetermined rules.

[0011] In the above, in an embodiment, the device further comprises aworking memory which temporarily pools and stores information from thesensing unit, the concern-generating unit, and the first-mentionedmemory until the device completes its action, and which outputsinformation to the concern-generating unit, the emotion-generating unit,the actuating unit, and the first-mentioned memory. Accordingly,processing becomes efficient.

[0012] The concern-parameters and the emotion-parameters can have anylabels and definitions. For example, the concern-parameters canrepresent “safety”, “affection”, “hunger”, and “play”, respectively, andthe emotion-parameters can represent “happy”, “angry”, “surprised”,“sad”, “fearful”, and “disgust”, respectively. The term “needs” can be asynonym for the concerns. The definition of each label can bepredetermined. The number of concern-parameters and the number ofemotion-parameters are not limited. The larger the number of parameters,the more complex the behavior becomes. If an object-oriented languageprogram, such as JAVA and C++, is used for parameter control, new labelscan be automatically created, so that new behavior can be generatedaccordingly.

[0013] A device controlled by the above control system can behaveautonomously and improve behavior based on its experience.

[0014] The present invention can be applied to a method as well as anapparatus. Further, the system need not be an integrated system, but canbe composed of plural separate units. That is, by networking pluraldevices or by using a separable medium (CD or IC card), an individualdevice can be downsized without losing memory and programmingcapacities.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015]FIG. 1 is a diagram showing an embodiment of the entire controlsystem according to the present invention.

[0016]FIG. 2 is a diagram showing data flow and processing flow of anembodiment according to the present invention.

[0017]FIG. 3 is a flow chart diagram showing an embodiment of thecontrol system according to the present invention.

[0018]FIG. 4 is a diagram showing an embodiment of a flow chart showingtime management flow in the present invention.

[0019]FIG. 5 is a diagram showing an embodiment of a neural network forthe concern-generating unit in the present invention.

[0020]FIG. 6 is a diagram showing an embodiment of schematic processesof JAVA in the present invention.

[0021]FIG. 7 is a diagram showing an embodiment of an emotion-generatingunit using a neural network in the present invention.

[0022]FIG. 8 is a diagram showing an embodiment of behavior patterns inthe present invention.

[0023]FIG. 9 is a diagram showing an embodiment of a behavior controlsystem in the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0024] The control system for controlling a device usingemotion-parameters of the present invention (referred to as “controlsystem”) will be explained with reference to embodiments indicated inthe figures.

[0025] Basic Control System

[0026] In an embodiment of the present invention, an agent or devicecomprises: (i) an external/internal condition recognition unit, (ii) aconcern-generating unit, (iii) an emotion-generating unit, (iv) along-term memory, (v) a working memory, (vi) a behavior planning unit,and (vii) an actuating unit. The above is a conceptual or functionalstructure, and need not be a physical structure; that is, these unitsneed not be separately provided, but can be integrated. Further, eachunit can be disposed in a device, but can also be disposed separatelyfrom the device.

[0027]FIG. 1 is a diagram showing an embodiment of the entire controlsystem according to the present invention. As described above, this is aconceptual or functional structure, but not a physical structure.

[0028] In the embodiment of FIG. 1, an autonomic device comprises asensing unit 1, a recognition unit 2, an emotion-generating unit 3, aconcern-generating unit 4, a working memory 5 for producing data sheets,a long-term memory 6 for storing data, a behavior planning unit 7, andan actuating unit 8. The sensing unit 1 senses various signals includingexternal and internal information. External information includes visual,tactile, auditory information, and electrically transmitted signals, andinternal information includes battery level and malfunction signals. Thesensed signals are inputted into the recognition unit 2 which analyzesthe signals to generate meaningful information which includescharacterization and identification of an event. This recognizedinformation is inputted into the working memory 5 for producing datasheets. The concern-generating unit 4 is programmed to generateconcern-parameters in response to the recognized information. Theconcern-parameters are not limited and may include “safety”,“affection”, “hunger”, “study”, and “play”. The recognized informationneed not be all information but at least a piece of information(pre-selected) recognized at the recognition unit 2. Eachconcern-parameter is regulated independently of or harmoniously witheach other. Each concern-parameter may include a time factor; that is,the value of each concern-parameter changes with time (some of them fadewith time while others increase with time). Further, eachconcern-parameter may have its own predetermined function and fluctuateaccordingly. The concern-generating unit 4 receives a signal from theworking memory 5, and outputs a signal of concern to the working memory5 which exchanges data with the long-term memory 6. Theemotion-generating unit 3 is programmed to generate emotion-parametersin response to the concern-parameters retrieved from the long-termmemory 6 via the working memory 5. This recognized information need notbe all but part of the information (pre-selected) recognized at therecognition unit 2, and may be different from that used at theconcern-generating unit 4. The emotion-parameters are not limited butmay include “happy”, “angry”, “surprised”, “sad”, “fearful”, and“disgust”. Each emotion-parameter is regulated independently of orharmoniously with each other. Each emotion-parameter may include a timefactor; that is, the value of each emotion-parameter changes with time(some of them fade with time while others increase with time). Further,the emotion-parameters are generated when receiving information from theworking memory 5. The emotion-generating unit 3 outputs signals ofemotions to the working memory 5 for producing data sheets. Thelong-term memory 6 is for storing data when the working memory 5produces a final data sheet and inputs it to the long-term memory 6. Thefinal data sheets are pooled and rearranged under the index of theobject in the long-term memory 6 so that data can be retrieved asnecessary. The data from the working memory 5 updates the long-termmemory 6. In an embodiment, the working memory 5 can be omitted, and thelong-term memory 6 can also function as the working memory 5, or viseversa. In another embodiment, the working memory 5 can be omitted, andthe long-term memory 6 can also function as the working memory 5, orvise versa. In another embodiment, the working memory 5 can be omitted,and each of the emotion-generating unit 3 and the concern-generatingunit 4 can include a memory and can be connected to each other and tothe other units.

[0029] As described above, the working memory 5 receives signals fromthe recognition unit 2, the emotion-generating unit 3, theconcern-generating unit 4, and the long-term memory 6 to produce datasheets (algorithms will be explained later). In the working memory 5, adata sheet may indicate an event, which includes an object and currentsituation surrounding the device recognized by the recognition unit 2,current concerns, which are generated by the concern-generating unit 4,and current emotions, which are generated by the emotion-generating unit3. Variations in the concern-parameters when previously encountering theevent, which are extracted from the long-term memory 6, can be but neednot be indicated in the data sheet in the working memory 5. Variationsin the concern-parameters stored in the long-term memory 6 areeffectively used for finding an event to best compensate for the currentvariations in the concern-parameters, so that the device can select anaction which may be most appropriate in the situation to modify theconcern-parameters. Based on these data, the behavior-planning unit 7decides on a pattern of behavior and outputs a signal to the actuatingunit 8. At the behavior planning unit 7, a behavior pattern may bemodified based on current emotions which are obtained by theemotion-generating unit 3; that is, behavior is selected based on thedata on the event extracted from the long-term memory 6, and is modifiedbased on the current emotions. Patterns of behavior are not limited butmay include “approach”, “attack”, “avoid”, “explore”, and “random”. Ifthe device has various movable parts such as a tail, ears, hands, andeyebrows, various patterns of behavior can be formed. In the above, fora simple device, behavior can be selected without referring to thecurrent emotions, and the emotion-generating unit 3 can be omitted.

[0030] Upon action by the actuating unit 8, the result is sensed by thesensing unit 1; that is, the result is fed back to the device. Forexample, as a result of approaching the object, if the device was hit,that impact is sensed by the sensing unit 1 and inputted into theworking memory 5 via the recognition unit 2 where the impact isconverted to variations in its concerns. The data sheet for this eventin the working memory 5 is complete upon inputting the variations in itsconcerns. This data sheet is provided to the long-term memory 6 andupdates the data therein. After one data sheet is complete (one event isover) and stored in the long-term memory 6, the device proceeds to a newevent.

[0031] Basic Data Flow

[0032]FIG. 2 is a schematic diagram showing data flow and processingflow of an embodiment according to the present invention. In thisfigure, first, the external/internal environment recognition unit(recognition unit) 2 outputs information (e.g., CONTACT=TOUCH; IR(infrared sensor)=NONE; VISION=USER A) to the working memory 5 whichwrites the information on a data sheet 10 a. The information is derivedfrom the sensing unit 1 and shows that the device is being touched,there is no obstacle in front of the device, and the device is seeinguser A. The data sheet 10 a includes items “CONTACT”, “IR”, “VISION”,“OBJECT OF INTEREST”, “CURRENT CONCERN”, and “CURRENT EMOTION”. Theseitems are simply an example and can be selected depending on the typesof sensors and recognition systems. In the data sheet 10 a, l “CONTACT”,“IR”, and “VISION” are updated using the information (current status)from the recognition unit 2.

[0033] Second, the current status is inputted into aninterest-generating unit 9. The interest-generating unit 9 is programmedto select an object of interest based on the information from a datasheet 10 a in the working memory 5. For a simple device, the objects canbe pre-selected, or can be defined by simple terms such as color,movement, and sound. That is, the “user” can be defined by motion,color, and voice, or simply by an electronic name tag. If theinterest-generating unit 9 does not find an object based on theinformation, the device behaves under predetermined rules such as“explore until an object is found” or “approach the object until thedevice's emotions change”. If the interest-generating unit 9 finds anobject, the object is designated temporarily as an object of interest,and the control system continues processing.

[0034] Third, the interest-generating unit 9 outputs the information(OBJECT OF INTEREST=USER A) to the working memory 5 which updates“OBJECT OF INTEREST” (data sheet 10 b). The data sheet 10 b and the datasheet 10 a are the same, but simply for the purpose of showing changeswith time or by step, the data sheet 10 b is separately indicated (datasheets 10 c and 10 d are separately indicated for the same reason).

[0035] Fourth, the information (OBJECT OF INTEREST=USER A) is inputtedto the long-term memory 6. The long-term memory 6 has a spreadsheetwherein data are stored under the indexes of objects. Various datastorage systems can be adapted in the long-term memory. Here, the memorystores data after every event. From the long-term memory 6, data storedunder the index of user A are extracted. Extraction can be conducted invarious ways. If the data sheet includes more information or items, datacan be extracted under plural indices. For example, data can becollected under indices of “USER A”, “TOUCH”, and “APPROACH”. Here, asingle index, “USER A”, is used. Under the index of “USER A”, variationsof concerns on record are totaled and averaged (or processed underpredetermined rules such as “the older the record, the less thecontribution becomes”). Each concern-parameter can be determinedseparately from the other concern-parameters. In this figure, however,for simplifying explanation, a single scale is used; that is,“SOCIALITY” scores are determined. For example, as a result ofcollecting data under the index of “USER A”, if a variation in“SOCIALITY” is calculated at 0.4, the score is added to the currentconcern score.

[0036] Fifth, the calculated variation is inputted into theconcern-generating unit 4 which generates a current concern of“SOCIALITY 0.1”. The score 0.4 is added to the current concern,resulting in “SOCIALITY 0.5”. There are various ways to determine“CURRENT CONCERNS” other than addition. The initial current emotion mayhave a time factor, i.e., changing with time (usually fading with time).The concern-generating unit 4 is programmed to generateconcern-parameters (explained later), and here, “sociality” has thehighest intensity among the concern-parameters (e.g., “sociality”,“safety”, “curiosity”, etc.). If the highest current concern is notsociality, user A may not be a good object of interest based on thevariations of concerns recorded in the long-term memory 6. In that case,as explained below, the emotion-generating unit 3 generates negativevalues of emotions, and accordingly, the device is actuated to ignoreuser A or move away from user A to find an appropriate object.

[0037] Sixth, the information (SOCIALITY=+0.5) is inputted into theworking memory 5 which updates “CURRENT CONCERN” under predeterminedrules (data sheet 10 c).

[0038] Seventh, the information (SOCIALITY=+0.5) is inputted into theemotion-generating unit 3. Here, the current emotion was initially“NEUTRAL” (“HAPPY” 0 (zero)). The emotion-generating unit 3 isprogrammed to generate emotion-parameters based at least on the currentconcern-parameters and the updated concern-parameters (explained later).Here, the “SOCIALITY” score increases of 0.4, and the emotion is changedfrom “NEUTRAL” to “HAPPY 0.5”. The initial current emotion may have atime factor, i.e., changing with time (usually fading with time).

[0039] Eighth, the information (HAPPY=+0.5) is inputted into the workingmemory 5 (data sheet 10 d). Here, the current emotion was initially“HAPPY 0” and changed to “HAPPY 0.5”.

[0040] Ninth, the behavior-planning unit 7 receives the information ofthe data sheet 10 d from the working memory 5. Here, the informationindicates that the device is being touched, there is no obstacle infront of the device, the device is seeing user A, and the emotion level(happy) increases of 0.5. The behavior-planning unit 7 is programmed todetermine a pattern of behavior based on these pieces of information, byreferring to a behavior-planning library 11 (step ten). Here, thebehavior-planning unit 7 selects “APPROACH” based on the situation thatthe emotion level increases of 0.5, and the direction is toward user A.The approaching speed may be dependent on the emotion level (e.g., thehigher the level, the higher the speed becomes).

[0041] Eleventh, the behavior-planning unit 7 outputs the information(BEHAVIOR=APPROACH, etc.) to the actuating unit 8. The actuating unit 8receives information from the behavior-planning unit 7 or directly fromthe working memory 5, so that the actuating unit 8 can determine thedirection of movement to achieve the task (APPROACH to USER A). Here,the actuating unit 8 determines “FORWARD”. As a result, the devicefurther approaches user A while user A is touching the device.

[0042] Twelfth, the device receives a consequence of the action, andsaves it in the long-term memory 6 as a new event.

[0043] In the figure, many flows are omitted. For example, theconcern-generating unit 4 may receive the information (CONTACT=TOUCH,etc.) from the working memory 5 to generate the current concerns and todetermine a satisfaction level of each concern. Further, theemotion-generating unit 3 may receive the information from theconcern-generating unit 4 and/or the recognition unit 2 in order togenerate emotion-parameters.

[0044] Basic Flow Chart

[0045]FIG. 3 is a flow chart diagram showing an embodiment of thecontrol system according to the present invention. As shown in FIG. 3,the device senses the external and internal environment (step a). Thecontrol system checks whether an object is recognized (step b). If anobject is observed, the object is selected as an object of interest(step c). Subsequently, if the object is found irrelevant, the deviceignores the object, but in step c, the device can assume that the objectis an object of interest for simple processing. Under the index of theobject, information on concern values is extracted from the long-termmemory j (step d). Based on the information, the current concernsgenerated by the concern-generating unit are modified (step e). Based onthe modified current concerns, the current emotions generated by theemotion-generating unit are modified (step f). Based on the modifiedemotions, the behavior-planning unit plans behavior by referring to thebehavior library k (step g). The device is actuated based on theinformation from the behavior-planning unit (step h). In step b, if noobject is found, the behavior-planning unit generates commands based onthe current concerns and/or emotions under predetermined rules (step g).The device is then actuated (step h). As a result, the deviceexperiences consequences of the device's behavior. The consequence issensed by the sensing unit and recognized by the recognizing unit, andthen the data are recorded as a new event (if no previous relevant eventwas found) or the data updates the previous data (if a relevant eventwas found) (step i).

[0046] Sensing and Recognition Units

[0047] In an embodiment, the device may comprise a CCD camera as avisual detection means, a pressure-sensing sensor and anapproach-sensing sensor as touch detection means, a microphone as ahearing-detection means, and an infrared sensor as an obstacle-detectionmeans. The device may further comprise a battery capacity sensor whichdetects the remaining capacity of the battery. Alternatively, the devicemay comprise a radio wave signal receiver. By using these sensing means,the device detects objects, the environment, and internal conditions.Further, these sensors allow the device to detect the state of the user,such as a tone of voice, facial expressions, and gestures, and theoperational environments where the device is used.

[0048] If the device is a robot, the CCD camera is installed on the topof the head and can be set in any direction via a universal joint. Forexample, the robot can be controlled in such a way that the robotautomatically moves toward an object, such as a human or animal, whichis a cause or source of information such as changes in temperature andsound. Image information such as facial expressions of the user andsurrounding environments is supplied to a controller.

[0049] The pressure-sensing sensor may be installed in the lower frontof the robot so that when the robot has actual contact with an obstacle,such information is provided to the controller.

[0050] The microphone may be installed on the side of the head of therobot, and provides information to the controller upon collectingsound/voices arose around the robot.

[0051] In the present invention, the sensed signals, i.e., primitiveinformation, can be used directly, without further processing. Forexample, if a color is sensed, and if the control system is designed towork based on the color (an object is recognized simply by the color),no further processing is required. However, if the device is designed torespond to more than the color, the device needs to recognize morecomplex information and may require processing information. For example,based on the color, movement, and sound, the object can be recognized.Further, if complex recognition systems are used, the user's facialexpression can be detected, and the device can respond to the emotionsof the user which are represented by the facial changes (Hanaya, et al.,“An attempt of individual identification from face photographs”,Technical Report of the Institute of Electronics, Information andCommunication Engineers, CS96-123, IE96-92 (1996-12), pp. 55-60). A faceneuron or neural network technology can be adapted.

[0052] In the present invention, for the purpose of simpleexperimentation, by using a radio wave detection sensor, the device candetect an object which possesses an identification tag transmittingidentification information. The radio wave detection sensor can transmitradio waves and receive resonance signals emitted from theidentification tag which has a particular resonance circuit. Inaddition, magnetic identification or bar code identification can beadapted. Further, a neural network can be used for recognizing an objectbased on the sensed signals.

[0053] Concern-Generating Unit

[0054] The concern-generating unit is used for selecting an object ofconcern, so that the device can behave and collect data efficiently, andcan improve its behavior quickly, without information overflow. Thedevice can select an object by itself, without a user's command, so thatthe device can behave autonomously. The concern-generating unit can beindependent from the external conditions or environment, and can be afunction of its own equations. However, the concern-generating unit maybe affected by the internal conditions such as battery level and thedegree of satisfaction of each concern-parameter. Further, theconcern-generating unit may have a time factor and change eachconcern-parameter with time. A change with time may occur in thedirection of fading. The functions controlling each concern-parametercan be independent of each other, and they may create fluctuation cyclesof each concern-parameter. For example, concern in “play” may fluctuateat predetermined intervals. The working memory may store data showingthe number of occurrences of “play” and a degree of satisfaction, sothat the concern-generating unit receives the data and modifies eachconcern-parameter. The working memory shown in FIG. 3 can include theitem “capacity” which shows a level or degree of satisfaction, and, forexample, when the consequent emotion is positive, the working memoryoutputs a signal to the concern-generating unit to raise thesatisfaction level or degree. In general, the concerns are positioned ata higher level than the emotions, and the emotions are controlled by theconcerns. However, the emotions can be one of the factors that modifythe concerns. If each concern-parameter is metaphorically represented bycontents in a tank, its satisfaction level can be represented by thelevel of the contents. The tank has two level switches; a first switchis a lower switch disposed at a lower part of the tank, which triggersopening a valve to fill the tank with the contents (concern-parameter),and the other switch is a higher switch disposed at a higher part of thetank, which triggers closing the valve to stop filling the tank with thecontents (concern-parameter). Even if the tank is full, the contentsleak slowly with time, and eventually the lower switch will beactivated. While the valve is open, new contents are introduced into thetank every time “sleep” is satisfied. The above mechanism can readily berealized in a program. If there are a plurality of concern-parameters, aplurality of tanks as above can be used, and the filling speed, theleaking speed, the positions of the lower switch and the higher switchcan vary among the tanks (concern-parameters). As described above, theconcern-generating unit can readily be preprogrammed.

[0055] In another embodiment, changes with time can be achieved as shownin FIG. 4 which is a flow chart showing time management flow. In FIG. 4,if there is any signal stimulating “safety” (step i), the value of theconcern-parameter is modified in accordance with the intensity of thestimulation (step ii). If there is no signal stimulating “safety” instep i, the value of the concern-parameter is modified underpredetermined rules (step iii). In FIG. 4, the value of theconcern-parameter is fading or is multiplied by 0.9.

[0056] In another embodiment, the concern-generating unit can beestablished using a neural network by regulating a relationship betweenthe recognized signals and the concern-parameters. FIG. 5 is a diagramshowing an embodiment of a neural network 30 for the concern-generatingunit. When using the neural network, each concern-parameter is relatedto one another. In addition, output of the neural network is connectedto a time management sub-routine 31 to change each output with time. Thetime management can be conducted by activating the time factor (e.g.,reducing 10%) at pre-selected time intervals or after every (orpre-selected number) running cycle(s) of a program.

[0057] The neural network can be established off-line or on-line. If itis conducted on-line, output of the device is fed back to the controlsystem, and coupling coefficients can be adjusted. Further, couplingcoefficients can be modified using evolutionary computing technologysuch as genetic algorithms and genetic programming. However, if on-lineestablishment is conducted, a “training” period will be required.

[0058] Further, the concern-generating unit can be constituted by amulti-dimensional map defined by the recognized signals and theconcern-parameters.

[0059] The concern-generating unit may select one concern-parameterwhich has the lowest value among all of the concern-parameters at thetime the concern-generating unit is called for. Selection of aconcern-parameter can be conducted in various ways, including selectionat random or under predetermined rules or functions.

[0060] Object Recognition

[0061] When the device has a new experience, that is, when no event isextracted from the long-term memory, the device can categorize an objector situation in order to record the new experience in the long-termmemory. However, if the device observes a new object, the device cannotrespond to it. For example, if “user B” is observed, but the device hasnever interacted with user B, the device may not recognize user A as apotential object. If the objects are defined by genetic characteristicsor loose rules such as a “moving” object making “sound”, user B can bean object of concern. After the device observes user B and while thedevice interacts with user B, the device can collect data onidentification of user B by using the sensing unit. The data can bestored in the long-term memory and written in the spreadsheet under theindex of user #2. Subsequently, when the device observes user #2 inputsdata on a data sheet in the working memory, the device extracts data onuser #2 from the long-term memory.

[0062] By using object-oriented languages such as JAVA and C++, thenumber of objects of concern can be increased autonomously. For example,in JAVA, first, “objects” are defined by “parameters” and “methods”, andthen, instances are defined based on “parameter” and “method”. FIG. 6 isa diagram showing schematic processes of JAVA. In this figure, the“object” is “user”, and “user parameters” are “color” and “voice”, and“method” is (user parameter×(color+voice)). Here, the program stores twoinstances; user #1 (color=−1 (red), voice=+1(high)), and user #2(color=+1(blue), voice=−1(low)). When the device observes user #3(color=0(yellow), voice=0(intermediate)), the program checks whetheruser #3 falls within the profiles of user #1 or #2. Here, user #3 isneither red (−1) nor blue (+1), indicating that user #3's color is newwhich is 0 (yellow). The program creates a new instance under the indexof user #3 and defines user #3 by color (0, yellow) and voice (0,intermediate). When the device does not observe an object of concern,the device may select an object which the device is seeing, as an objectof concern as explained above. User #3 will be one of objects ofconcern. That is, when the concern-generating unit selects “play”, theprogram checks whether the object the device is seeing is either one ofusers #1, #2, and #3. By continuing such experiments, the device storesmany objects of concern.

[0063] Emotion-Generating Unit

[0064] Emotions generated by the device may be used for modifying thebehavior selected by extracting data from the long-term memory. Theemotion-generating unit can be operated by various ways which include aneural network. FIG. 7 is a diagram showing an embodiment of anemotion-generating unit using a neural network. The structure of theneural network in FIG. 7 is similar to that indicated in FIG. 5. Thedifference is that, in addition to pre-selected sensed signals (such asimpact and sound), this neural network uses, as input, signals of thecurrent concerns (such as safety, curiosity, and hunger) generated bythe concern-generating unit and the modified concerns obtained based onthe information extracted from the long- term memory. Accordingly, ifthere is a negative discrepancy between the current concerns and themodified concerns, the device may develop negative emotions such as“sad” and “angry”, which negative emotions affect the device's behavior.For example, if the current concern is hunger (unsatisfied=a negativevalue), but the object the device is observing is user A who will notsatisfy the device as to the concern of hunger (i.e., no positive valuewill be provided), the device may develop the emotion of sadness oranger, thereby avoiding the object to find another object. If thecurrent concern is sociality (unsatisfied=a negative value), and theobject the device is observing is user A who will satisfy the device asto the concern of sociality (i.e., a positive value will be provided),the device may develop the emotion of happiness, thereby approaching theobject. The input-output relationship of the neural network may beregulated in advance by off-line training, although the neural networkcan be established off-line or on-line. If it is conducted on-line,output of the device is fed back to the control system, and couplingcoefficients can be adjusted. Further, coupling coefficients can bemodified using evolutionary computing technology such as geneticalgorithms and genetic programming. However, if on-line establishment isconducted, a “training” period will be required.

[0065] For establishing an emotion generation system, the technologydisclosed in U.S. patent application Ser. No. 09/059,278, filed Apr. 13,1998, by Tamiya, et al., entitled “CONTROL SYSTEM FOR CONTROLLING OBJECTUSING PSEUDO-EMOTIONS GENERATED IN THE OBJECT”, can be adapted to thepresent invention. The reference is hereby incorporated herein asreference. In the above, each emotion-parameter is defined by a facialexpression which the robot can output, and teacher data are not requiredto establish the neural network if output evaluation can be conducted byusing reward signals (e.g., being caressed) or penalty signals (e.g.,being hit), instead of teacher data.

[0066] Further, the emotion-generating unit can be constituted by amulti-dimensional map defined by the recognized signals and theemotion-parameters.

[0067] For a very simple model, the input-output relationship in theemotion-generating unit can be simplified by directly relatingvariations in the concerns with emotions, without the sensed signals. Adiscrepancy between the current concerns and the modified concerns canbe determined by the least square method.

[0068] Behavior-Planning Unit

[0069]FIG. 8 is a diagram showing an embodiment of behavior patterns(behavior library). As a result of variations in the concerns, theemotions are modified, and behavior will be selected. For a highlysimplified device, the behavior-planning unit can be very simple. Forexample, the following conditions can be predetermined: Modified ConcernBehavior Sociality Approach Safety Avoid Fatigue Sleep Curiosity Explore. . . . . .

[0070] The selected behavior can be modified by variations in itsemotions. That is, the device's behavior is modified in accordance withthe variations in the emotions. If the initial behavior is “APPROACH”,when the device raises an “ANGRY” score, the behavior is changed to“ATTACK”; when the device raises a “FEAR” score, the behavior is changedto “AVOID”; when the device raises a “DISGUST” score, the behavior ischanged to “RANDOM”; when the device raises a “SAD” score, the behavioris changed to “EXPLORE”; when the device raises a “HAPPY” score, thebehavior is not changed, but may be more active (faster movement). Ifthe initial behavior is “ATTACH”, when the device raises a “HAPPY”score, the behavior is changed to “APPROACH”; when the device raises a“DISGUST”score, the behavior is changed to “RANDOM”; and when the deviceraises an “ANGRY” score, the behavior is not changed, but may be moreactive (faster movement). When the device avoids the object, the deviceraises an “ANGRY” score (run away is not successful), the behavior ischanged to “ATTACK”. In the long-term memory, the action which isactually performed is recorded and linked with the other data under theindex of the situation.

[0071] These behavior patterns can be established by a neural network byregulating the input-output relationship or by a multi-dimensional map.Further, coupling coefficients of the neural network can be modifiedusing evolutionary computing technology such as genetic algorithms andgenetic programming. However, if on-line establishment is conducted, a“training” period will be required. For establishing behavior patterns,the technology disclosed in U.S. patent application Ser. No. 09/059,278,filed Apr. 13, 1998, by Tamiya, et al., as described above, can beadapted to the present invention. The reference is hereby incorporatedherein as reference.

[0072] Further, the behavior-planning unit can be constituted by amulti-dimensional map defined by the generated concern-parameters andthe generated emotion-parameters. In the above, if the device has aplurality of moving parts, all moving parts can be allocated on amulti-dimensional map. This approach can be used in combination with aneural network, wherein a multi-dimensional map is connected downstreamof the neural network. For example, sensed signals, emotion-parameters,and concern-parameters (these data are stored in the working memory) areused as input signals of the neural network. “APPROACH”, “ATTACK”,“EXPLORE”, “AVOID”, “RANDOM”, “STILL”, etc. are outputted from theneural network. Training the neural network can be conducted off-line.Downstream of the neural network, a multi-dimensional map is connected,which regulates the output in such a way, for example, that if“APPROACH” 0.1 and “EXPLORE” 0.3, the device slowly moves toward theobject with a tail slowly moving. Not only the neural network but alsothe map can be modified by using evolutionary computing technology.Evolutionary reinforcement learning methods does not require teacherdata. Evaluation functions can be obtained by using reward signals(e.g., being caressed) or penalty signals (e.g., being hit), instead ofteacher data.

[0073]FIG. 9 is a diagram showing an embodiment of a behavior controlsystem. The behavior-planning unit can have a plurality of modules, eachregulating one action. The modules can be selected by using technologydescribed above. Each module commands the device's movement sequentiallyon conditions of the external and internal conditions. FIG. 9 shows anembodiment of a “RUN AWAY” module. If the object is behind the device,the device moves forward (action 1). If the object is in front of thedevice, the device turns around (action 2). If predetermined conditionsare satisfied, action 1 (moving forward) is triggered from action 2. Atrigger may be defined by the distance from the object, the moving speedof the object and its direction, the intensity of sound the object ismaking, the absence of obstacles, etc. The trigger can be determinedbased on theoretical sum of all of the conditions, each having a triggerrange. If the device moves a certain distance, the device stops (action3). If the device stops, the device turns at 180 degrees withoutcondition (action 4). If the object is close to the device, action 2 istriggered. If the object is not close to the device, the series ofactions end. In the above, each action is constituted by a sub-module.Each sub-module can include emotion-parameters. For example, the modulefor action 1 (moving forward) can be programmed to change the movingspeed in accordance with emotion-parameters. If a “FEARFUL” score ishigh, the device moves faster than when the score is low. If the devicemoves away from the object, but the device is hit by the object, the“FEARFUL” score becomes high. These data are transferred to the workingmemory and update the data sheet. The data are saved in the long-termmemory. Subsequently, when the device faces the same situation andextracts the data from the long-term memory, the sub-module for action 1receives a signal to raise the moving speed (50). As described above,the spreadsheet in the long-term memory, the data sheet in the workingmemory, and the action modules can be modified based on the user'spreferences.

[0074] Other Features

[0075] In the above, if the device does not have sufficient capacity(processing capacity, data storing capacity), the connected toy can beconnected to an external system (computer) through cordless networks.That is, the data and/or programs used in the present system, includingthe long-term memory, the working memory, the emotion-generating unit,the concern-generating unit, and the behavior-planning unit, can besaved or installed in a separable medium such as a compact disc (ROM orRAM) or IC card, so that the user can implant the information in anotherdevice. In addition, the data and/or programs can be transmitted toanother device via communication means. By using the above technology,plural memory media of plural devices can be hybridized or cross bred tocreate a new system. The data can be pooled from plural devices andinstalled into a new device which will possess an extensive memory evenif the new device has never been used. By changing the programs,behavior patterns of the device can be changed. Further, any one or moreof the intelligent portions of the device including the recognitionunit, the emotion-generating unit, the concern-generating unit, and thebehavior-planning unit can be installed in a main computer separatelyfrom the device, wherein a network is established between the maincomputer and the device via, for example, the Internet, so that thedevice can be made compact, and which may need to have simply sensingunits, an output unit, and a communication unit for contacting the maincomputer. Through the sensing units of each device, it is possible tomonitor the user or the external environment of each device, by use ofthe main computer. Further, each device can be connected to otherdevices to establish a network of devices to exchange information.

[0076] Other Aspects

[0077] In the present invention, correlations between various inputs andvarious outputs of the control system can be determined using existingtechniques such as neural networks, fuzzy neural networks, and geneticalgorithms if the correlations are highly complex, or using existingtechniques such as maps and functional equations if the correlations arerather simple. In this regard, Da Ruan (editor)“Intelligent HybridSystems—Fuzzy Logic, Neural Networks, and Genetic Algorithms—” KluwerAcademic Publishers (1997), J. -S. R. Jang, C. -T. Sun, E.Mizutani,“Neuro-Fuzzy and Soft Computing” Prentice Hall Upper SaddleRiver, N.J. 07458 (1997), C. -T. Lin and C. S. George Lee, “Neural FuzzySystems” Prentice Hall Upper Saddle River, N.J. 07458 (1998), and N. K.Kasabov, “Foundations of Neural Networks, Fuzzy Systems, and KnowledgeEngineering” the MIT Press (1996) are hereby incorporated by reference.The above techniques can be combined, and learning control can beadapted for any technique.

[0078] Further, in addition to genetic algorithms (GA), geneticprogramming (GP) or other evolutionary computing techniques can beadapted to the present invention (Wolfgang Banzhaf, et al. (editor),“Genetic Programming, An Introduction”, pp. 363-377, 1999, MorganKaufmann Publishers, Inc., for example). These techniques are sometimescategorized as “heuristic control” which includes evolution, simulatedannealing, and reinforcement learning method (S. Suzuki, et al.,“Vision-Based Learning for Real Robot: Towards RoboCup”, RoboCup - 97Workshop, 23, 24, and 29 August, 1997 Nagoya Congress Center, pp.107-110; K. and Nurmela, et al., “Constructing Covering Designs BySimulated Annealing”, pp. 4-7, Helsinki University of Technology,Digital Systems Laboratory, Technical Reports No. 10, January 1993, forexample). These techniques can be adapted to the present inventionwithout complication, based on the principle described earlier; that is,in the present invention, “evolutionary computing” includes the abovevarious techniques.

[0079] Further, the evolutionary computing includes a multi-agent systemwhich is used for competitive co-evolution (Tamashiro, et al., “Studyabout the performance of competitive co-evolution in multi-agentsystem”, Technical Report of the Institute of Electronics, Informationand Communication Engineers, NS99-16 (1999-06), pp.37-41).

[0080] Further, in the above, neural networks may be used for learningcontrol rules. However, a CMAC (Cerebellar Model Arithmetic Computer)can also be used. The CMAC is excellent in terms of additional learningand the high speed of learning, as compared with the hierarchical neuralnetwork.

[0081] Other Applications

[0082] In the above, the device may be a personal robot, toy robot orrobot pet. However, the device of the present control system is notlimited to a toy robot, and can be any given control which can be usedin a vehicle, an auxiliary drive of a bicycle, or wheelchair, or anindustrial robot.

[0083] It will be understood by those of skill in the art that numerousand various modifications can be made without departing from the spiritof the present invention. Therefore, it should be clearly understoodthat the forms of the present invention are illustrative only and arenot intended to limit the scope of the present invention.

What is claimed is:
 1. A method for adjusting behavior of a device basedon the device's experience, said device comprising: (i) a sensing unitfor sensing signals; (ii) a concern-generating unit programmed togenerate concern-parameters; (iii) an emotion-generating unit programmedto generate emotion-parameters; and (iv) an actuating unit for actuatingthe device, said method comprising the steps of: (a) recognizing anobject based on signals receivable by the device, said device havinginitial concern parameters; (b) extracting information, if any, onconcern-parameters from a memory under at least the index of the object,said memory storing under the index of the object, information onconcern-parameters previously generated by the device through pastinteraction with the object; (c) modifying the initialconcern-parameters by the extracted information on concern-parameters;(d) actuating the device based on the modified concern-parameters andemotion-parameters generated by the device based on a discrepancybetween the current concern-parameters and the modifiedconcern-parameters under predetermined rules; and (e) inputting in thememory, under the index of the object, concern-parameters generated bythe device upon interaction with the object, thereby updating thememory.
 2. The method according to claim 1, wherein the behavior isselected based on the modified concern-parameters and is then modifiedbased on the emotion-parameters under predetermined rules.
 3. The methodaccording to claim 1, wherein, in step (b), if no information on theobject is extracted from the memory, the device behaves underpredetermined rules.
 4. The method according to claim 1, wherein theinput-output relationship of the concern-generating unit ispredetermined, said concern-generating unit receiving pre-selectedsensed signals and outputting the concern-parameters.
 5. The methodaccording to claim 1, further comprising a working memory whichtemporarily pools and stores information from the sensing unit, theconcern-generating unit, and the first-mentioned memory until the devicecompletes its action, and which outputs information to theconcern-generating unit, the emotion-generating unit, the actuatingunit, and the first-mentioned memory.